Application of Multi-Criteria Decision Analysis (MCDA) to Prioritize Real-World Evidence Studies for Health Technology Management: Outcomes and Lessons Learned by the Canadian Real-World Evidence for Value of Cancer Drugs (CanREValue) Collaboration.

Multi-criteria decision analysis (MCDA) is a value assessment tool designed to help support complex decision-making by incorporating multiple factors and perspectives in a transparent, structured approach. We developed an MCDA rating tool, consisting of seven criteria evaluating the importance and feasibility of conducting potential real-world evidence (RWE) studies aimed at addressing uncertainties stemming from initial cancer drug funding recommendations. In collaboration with the Canadian Agency for Drugs and Technologies in Health's Provincial Advisory Group, a validation exercise was conducted to further evaluate the application of the rating tool using RWE proposals varying in complexity. Through this exercise, we aimed to gain insight into consensus building and deliberation processes and to identify efficiencies in the application of the rating tool. An experienced facilitator led a multidisciplinary committee, consisting of 11 Canadian experts, through consensus building, deliberation, and prioritization. A total of nine RWE proposals were evaluated and prioritized as low (n = 4), medium (n = 3), or high (n = 2) priority. Through an iterative process, efficiencies and recommendations to improve the rating tool and associated procedures were identified. The refined MCDA rating tool can help decision-makers prioritize important and feasible RWE studies for research and can enable the use of RWE for the life-cycle evaluation of cancer drugs.

Development of a Multi-Criteria Decision Analysis Rating Tool to Prioritize Real-World Evidence Questions for the Canadian Real-World Evidence for Value of Cancer Drugs (CanREValue) Collaboration.

The Canadian Real-world Evidence for Value of Cancer Drugs (CanREValue) collaboration developed an MCDA rating tool to assess and prioritize potential post-market real-world evidence (RWE) questions/uncertainties emerging from public drug funding decisions in Canada. In collaboration with a group of multidisciplinary stakeholders from across Canada, the rating tool was developed following a three-step process: (1) selection of criteria to assess the importance and feasibility of an RWE question; (2) development of rating scales, application of weights and calculating aggregate scores; and (3) validation testing. An initial MCDA rating tool was developed, composed of seven criteria, divided into two groups. Group A criteria assess the importance of an RWE question by examining the (1) drug's perceived clinical benefit, (2) magnitude of uncertainty identified, and (3) relevance of the uncertainty to decision-makers. Group B criteria assess the feasibility of conducting an RWE analysis including the (1) feasibility of identifying a comparator, (2) ability to identify cases, (3) availability of comprehensive data, and (4) availability of necessary expertise and methodology. Future directions include partnering with the Canadian Agency for Drugs and Technology in Health's Provincial Advisory Group for further tool refinement and to gain insight into incorporating the tool into drug funding deliberations.

Cost-effectiveness of second-line ipilimumab for metastatic melanoma: A real-world population-based cohort study of resource utilization.

BACKGROUND: The efficacy-effectiveness gap between randomized trial and real-world evidence regarding the clinical benefit of ipilimumab for metastatic melanoma (MM) has been well characterized by previous literature, consistent with initial concerns raised by health technology assessment agencies (HTAs). As these differences can significantly impact cost-effectiveness, it is critical to assess the real-world cost-effectiveness of second-line ipilimumab versus non-ipilimumab treatments for MM. METHODS: This was a population-based retrospective cohort study of patients who received second-line non-ipilimumab therapies between 2008 and 2012 versus ipilimumab treatment between 2012 and 2015 (after public reimbursement) for MM in Ontario. Using a 5-year time horizon, censor-adjusted and discounted (1.5%) costs (from the public payer's perspective in Canadian dollars) and effectiveness were used to calculate incremental cost-effectiveness ratios (ICERs) in life-years gained (LYGs) and quality-adjusted life years (QALYs), with bootstrapping to capture uncertainty. Varying the discount rate and reducing the price of ipilimumab were done as sensitivity analyses. RESULTS: In total, 329 MM were identified (Treated: 189; Controls: 140). Ipilimumab was associated with an incremental effectiveness of 0.59 LYG, incremental cost of $91,233, and ICER of $153,778/LYG. ICERs were not sensitive to discounting rate. Adjusting for quality of life using utility weights resulted in an ICER of $225,885/QALY, confirming the original HTA estimate prior to public reimbursement. Reducing the price of ipilimumab by 100% resulted in an ICER of $111,728/QALY. CONCLUSION: Despite its clinical benefit, ipilimumab as second-line monotherapy for MM patients is not cost-effective in the real world as projected by HTA under conventional willingness-to-pay thresholds.

Comparison of Use of Neoadjuvant Systemic Treatment for Breast Cancer and Short-term Outcomes Before vs During the COVID-19 Era in Ontario, Canada.

Importance: In response to an increase in COVID-19 infection rates in Ontario, several systemic treatment (ST) regimens delivered in the adjuvant setting for breast cancer were temporarily permitted for neoadjuvant-intent to defer nonurgent breast cancer surgical procedures. Objective: To examine the use and compare short-term outcomes of neoadjuvant-intent vs adjuvant ST in the COVID-19 era compared with the pre-COVID-19 era. Design, Setting, and Participants: This was a retrospective population-based cohort study in Ontario, Canada. Patients with cancer starting selected ST regimens in the COVID-19 era (March 11, 2020, to September 30, 2020) were compared to those in the pre-COVID-19 era (March 11, 2019, to March 10, 2020). Patients were diagnosed with breast cancer within 6 months of starting systemic therapy. Main Outcomes and Measures: Estimates were calculated for the use of neoadjuvant vs adjuvant ST, the likelihood of receiving a surgical procedure, the rate of emergency department visits, hospital admissions, COVID-19 infections, and all-cause mortality between treatment groups over time. Results: Among a total of 10 920 patients included, 7990 (73.2%) started treatment in the pre-COVID-19 era and 7344 (67.3%) received adjuvant ST; the mean (SD) age was 61.6 (13.1) years. Neoadjuvant-intent ST was more common in the COVID-19 era (1404 of 2930 patients [47.9%]) than the pre-COVID-19 era (2172 of 7990 patients [27.2%]), with an odds ratio of 2.46 (95% CI, 2.26-2.69; P < .001). This trend was consistent across a range of ST regimens, but differed according to patient age and geography. The likelihood of receiving surgery following neoadjuvant-intent chemotherapy was similar in the COVID-19 era compared with the pre-COVID-19 era (log-rank P = .06). However, patients with breast cancer receiving neoadjuvant-intent hormonal therapy were significantly more likely to receive surgery in the COVID-19 era (log-rank P < .001). After adjustment, there were no significant changes in the rate of emergency department visits over time between patients receiving neoadjuvant ST, adjuvant ST, or ST only during the ST treatment period or postoperative period. Hospital admissions decreased in the COVID-19 era for patients who received neoadjuvant ST compared with adjuvant ST or ST alone (P for interaction = .01 for both) in either setting. Conclusions and Relevance: In this cohort study, patients were more likely to start neoadjuvant ST in the COVID-19 era, which varied across the province and by indication. There was limited evidence to suggest any substantial impact on short-term outcomes.

Real-World Cost-Effectiveness of First-Line Gemcitabine Plus Nab-Paclitaxel vs FOLFIRINOX in Patients With Advanced Pancreatic Cancer.

BACKGROUND: There are no randomized control trials (RCTs) comparing gemcitabine and nab-paclitaxel (Gem-Nab) and fluorouracil, folinic acid, irinotecan, oxaliplatin (FOLFIRINOX) for advanced pancreatic cancer (APC). Although it is well known that RCT-based efficacy often does not translate to real-world effectiveness, there is limited literature investigating comparative cost-effectiveness of Gem-Nab vs FOLFIRINOX for APC. We aimed to examine the real-world cost-effectiveness of Gem-Nab vs FOLFIRINOX for APC in Ontario, Canada. METHODS: This study compared patients treated with first-line Gem-Nab or FOLFIRINOX for APC in Ontario from April 2015 to March 2019. Patients were linked to administrative databases. Using propensity scores and a stabilizing weights method, an inverse probability of treatment weighted cohort was developed. Mean survival and total costs were calculated over a 5-year time horizon, adjusted for censoring, and discounted at 1.5%. Incremental cost-effectiveness ratio and net monetary benefit were computed to estimate cost-effectiveness from the public health-care payer's perspective. Sensitivity analysis was conducted using the propensity score matching method. RESULTS: A total of 1988 patients were identified (Gem-Nab: n = 928; FOLFIRINOX: n = 1060). Mean survival was lower for patients in the Gem-Nab than the FOLFIRINOX group (0.98 vs 1.26 life-years; incremental effectiveness = -0.28 life-years [95% confidence interval = -0.47 to -0.13]). Patients in the Gem-Nab group incurred greater mean 5-year total costs (Gem-Nab: $103 884; FOLFIRINOX: $101 518). Key cost contributors include ambulatory cancer care, acute inpatient hospitalization, and systemic therapy drug acquisition. Gem-Nab was dominated by FOLFIRINOX, as it was less effective and more costly. Results from the sensitivity analysis were similar. CONCLUSIONS: Gem-Nab is likely more costly and less effective than FOLFIRINOX and therefore not considered cost-effective at commonly accepted willingness-to-pay thresholds.

Mapping Canadian Data Assets to Generate Real-World Evidence: Lessons Learned from Canadian Real-World Evidence for Value of Cancer Drugs (CanREValue) Collaboration's RWE Data Working Group.

Canadian provinces routinely collect patient-level data for administrative purposes. These real-world data (RWD) can be used to generate real-world evidence (RWE) to inform clinical care and healthcare policy. The CanREValue Collaboration is developing a framework for the use of RWE in cancer drug funding decisions. A Data Working Group (WG) was established to identify data assets across Canada for generating RWE of oncology drugs. The mapping exercise was conducted using an iterative scan with informant surveys and teleconference. Data experts from ten provinces convened for a total of three teleconferences and two in-person meetings from March 2018 to September 2019. Following each meeting, surveys were developed and shared with the data experts which focused on identifying databases and data elements, as well as a feasibility assessment of conducting RWE studies using existing data elements and resources. Survey responses were compiled into an interim data report, which was used for public stakeholder consultation. The feedback from the public consultation was used to update the interim data report. We found that databases required to conduct real-world studies are often held by multiple different data custodians. Ninety-seven databases were identified across Canada. Provinces held on average 9 distinct databases (range: 8-11). An Essential RWD Table was compiled that contains data elements that are necessary, at a minimal, to conduct an RWE study. An Expanded RWD Table that contains a more comprehensive list of potentially relevant data elements was also compiled and the availabilities of these data elements were mapped. While most provinces have data on patient demographics (e.g., age, sex) and cancer-related variables (e.g., morphology, topography), the availability and linkability of data on cancer treatment, clinical characteristics (e.g., morphology and topography), and drug costs vary among provinces. Based on current resources, data availability, and access processes, data experts in most provinces noted that more than 12 months would be required to complete an RWE study. The CanREValue Collaboration's Data WG identified key data holdings, access considerations, as well as gaps in oncology treatment-specific data. This data catalogue can be used to facilitate future oncology-specific RWE analyses across Canada.

Cost-effectiveness Analysis of Pertuzumab With Trastuzumab in Patients With Metastatic Breast Cancer.

IMPORTANCE: The initial assessment of pertuzumab use for treatment of metastatic breast cancer by health technology assessment agencies suggested that pertuzumab was not cost-effective. In Ontario, Canada, pertuzumab became funded in November 2013 based on the substantial clinical benefit. To date, there is a paucity of analysis of pertuzumab using real-world data for cost-effectiveness. OBJECTIVE: To assess the cost-effectiveness of pertuzumab, trastuzumab, and chemotherapy vs trastuzumab and chemotherapy for patients with metastatic breast cancer. DESIGN, SETTING, AND PARTICIPANTS: A population-based retrospective economic evaluation was conducted in Ontario, Canada. Patients who received first-line treatments for metastatic breast cancer from January 1, 2008, to March 31, 2018, were identified. Patients were followed up from the start of treatment up to 5 years, with maximum follow-up to March 31, 2019. Patients were identified from the Ontario Cancer Registry and linked to the New Drug Funding Program database to identify receipt of first-line treatment (N = 1158). INTERVENTIONS: Treatment with pertuzumab, trastuzumab, and chemotherapy after public funding (November 25, 2013) compared with treatment with trastuzumab and chemotherapy before funding. MAIN OUTCOMES AND MEASURES: Cost-effectiveness, from a public payer perspective, was estimated from administrative data with a 5-year time horizon, adjusted for censoring, and discounted (1.5%). Incremental cost-effectiveness ratios for life-years gained and quality-adjusted life year (QALY) with bootstrapped 95% CIs were calculated. Sensitivity analysis with price reduction of pertuzumab alone or in combination with trastuzumab was conducted. RESULTS: A total of 579 pairs of matched patients receiving pertuzumab and controls were included. The mean (SD) age of the matched study cohort was 58 (12.97) years; 1151 were women (99.4%). Pertuzumab resulted in 0.61 life-years gained and 0.44 QALYs gained at an incremental cost of $192 139 (all costs measured in Canadian dollar values, CAD) with an incremental cost-effectiveness ratio of $316 203 per life-year gained and $436 679 per QALY. The main factors associated with cost included the cost of pertuzumab (60%), outpatient cancer treatment delivery (24%), and trastuzumab (15%). With 100% price reduction of pertuzumab, the incremental cost-effectiveness ratio was $174 027 per QALY. When the price of pertuzumab and trastuzumab were both reduced by more than 71%, the incremental cost-effectiveness ratio decreased below $100 000 per QALY. CONCLUSIONS AND RELEVANCE: The findings of this population-based study suggest that pertuzumab may increase survival for patients with metastatic breast cancer but would not be considered cost-effective, even after 100% price reduction, under conventional thresholds.

Comparative Effectiveness and Safety of Pertuzumab and Trastuzumab Plus Chemotherapy vs Trastuzumab Plus Chemotherapy for Treatment of Metastatic Breast Cancer.

IMPORTANCE: To date, limited studies have examined the comparative outcomes of pertuzumab treatment in the real-world setting. End-of-study analyses of the CLEOPATRA trial found median overall survival (OS) of 57.1 months in patients receiving pertuzumab compared with 40.8 months in control patients, a benefit of 16.3 months. However, studies examining the real-world use of pertuzumab have found conflicting results. OBJECTIVE: To assess the real-world comparative effectiveness and safety of pertuzumab, trastuzumab, and chemotherapy for patients with metastatic breast cancer in Ontario, Canada. DESIGN, SETTING, AND PARTICIPANTS: A population-based retrospective comparative effectiveness research study was conducted. Patients receiving first-line treatments for metastatic breast cancer from January 1, 2008, to March 31, 2018, in Ontario were identified. Data analysis was performed from November 13, 2019, to August 1, 2021. Thirteen patients had received treatment before diagnosis or were not Ontario residents and were excluded from the analysis. Of the remaining 1823 patients identified, 912 received pertuzumab and 911 were control patients. Using propensity-score methods, 579 pairs of patients receiving pertuzumab were matched to those in the control group, resulting in a total of 1158 patients in the final cohort. EXPOSURES: Patients in the case group received pertuzumab with trastuzumab and chemotherapy and those in the control group received trastuzumab and chemotherapy. MAIN OUTCOMES AND MEASURES: Overall survival (the primary outcome) and hazard ratios (HRs) were calculated using Kaplan-Meier and Cox proportional hazards regression methods. Secondary outcomes included cumulative incidence of safety end points including resource use and adverse events. Follow-up duration was up to 5 years from the start of therapy, with maximum follow-up to March 31, 2019. RESULTS: Of the 1158 matched patients (579 pairs) receiving pertuzumab and controls, 1151 (99%) were women (mean [SD] age, 58.2 [12.97] years). The median OS was higher in patients receiving pertuzumab (40.2; 95% CI, 35.6-47.8 months) than in the control patients (25.3; 95% CI, 22.8-27.6 months), a median OS improvement of 14.9 months. Pertuzumab was associated with reduced mortality (HR, 0.66; 95% CI, 0.57-0.79). The cumulative incidence of direct hospitalization at 1 year was lower among patients receiving pertuzumab (11.7%) compared with the control patients (19.0%) (P < .001). CONCLUSIONS AND RELEVANCE: Although the median OS in both the pertuzumab and control groups were shorter in this study than those observed in the CLEOPATRA trial, there appears to be a similar significant OS benefit with pertuzumab in the real-world setting.

Building a National Reassessment Process for Oncology Drugs: Lessons Learned by the Canadian Real-World Evidence for Value of Cancer Drugs (CanREValue) Collaboration through a Simulated Reassessment Exercise.

The CanREValue Collaboration established the Reassessment & Uptake Working Group to develop a preliminary process to reassess funded cancer drugs in Canada. A simulated exercise was conducted to evaluate the proposed reassessment process using a real-world case. We invited 32 attendees including representatives from Health Canada and Health Technology Assessment (HTA) agencies, along with payers, clinicians, academics, and patient representatives. A case was developed using a real-world study on a publicly funded cancer drug. In facilitated group sessions, participants were asked to deliberate upon the evidence presented in the case to issue reassessment recommendations. Several themes were identified through the deliberation discussions. While the generalizability of real-world evidence (RWE) is perceived as a strength, trust in the RWE depends largely on the source of the real-world data. The attendees suggested several improvements to the proposed reassessment process including evidence requirement for reassessment, recommendation categories, and a priori study protocols. This exercise generated important insights on the evidence required for conducting reassessment and considerations for improvements of the proposed reassessment process. Building upon lessons from this exercise, future work would continue to refine the reassessment process as part of the overall CanREValue framework.

Impact of limited sample size and follow-up on single event survival extrapolation for health technology assessment: a simulation study.

INTRODUCTION: Extrapolation of time-to-event data from clinical trials is commonly used in decision models for health technology assessment (HTA). The objective of this study was to assess performance of standard parametric survival analysis techniques for extrapolation of time-to-event data for a single event from clinical trials with limited data due to small samples or short follow-up. METHODS: Simulated populations with 50,000 individuals were generated with an exponential hazard rate for the event of interest. A scenario consisted of 5000 repetitions with six sample size groups (30-500 patients) artificially censored after every 10% of events observed. Goodness-of-fit statistics (AIC, BIC) were used to determine the best-fitting among standard parametric distributions (exponential, Weibull, log-normal, log-logistic, generalized gamma, Gompertz). Median survival, one-year survival probability, time horizon (1% survival time, or 99th percentile of survival distribution) and restricted mean survival time (RMST) were compared to population values to assess coverage and error (e.g., mean absolute percentage error). RESULTS: The true exponential distribution was correctly identified using goodness-of-fit according to BIC more frequently compared to AIC (average 92% vs 68%). Under-coverage and large errors were observed for all outcomes when distributions were specified by AIC and for time horizon and RMST with BIC. Error in point estimates were found to be strongly associated with sample size and completeness of follow-up. Small samples produced larger average error, even with complete follow-up, than large samples with short follow-up. Correctly specifying the event distribution reduced magnitude of error in larger samples but not in smaller samples. CONCLUSIONS: Limited clinical data from small samples, or short follow-up of large samples, produce large error in estimates relevant to HTA regardless of whether the correct distribution is specified. The associated uncertainty in estimated parameters may not capture the true population values. Decision models that base lifetime time horizon on the model's extrapolated output are not likely to reliably estimate mean survival or its uncertainty. For data with an exponential event distribution, BIC more reliably identified the true distribution than AIC. These findings have important implications for health decision modelling and HTA of novel therapies seeking approval with limited evidence.

The Association of Drug-Funding Reimbursement With Survival Outcomes and Use of New Systemic Therapies Among Patients With Advanced Pancreatic Cancer.

Importance: Gemcitabine-nab-paclitaxel (GEMNAB) and fluorouracil, leucovorin, irinotecan, and oxaliplatin (FOLFIRINOX) both improve survival of patients with advanced pancreatic cancer when compared with single-agent gemcitabine in clinical trials. Objective: To describe changes in the survival of patients with advanced pancreatic cancer associated with sequential drug-funding approvals and to determine if there exist distinct patient populations for whom GEMNAB and FOLFIRINOX are associated with survival benefit. Design, Setting, and Participants: This population-based, retrospective cohort study examined all incident cases of advanced pancreatic cancer treated with first-line chemotherapy in Ontario, Canada (2008-2018) that were identified from the Cancer Care Ontario (Ontario Health) New Drug Funding Program database. Statistical analysis was performed from October 2020 to January 2021. Exposures: First-line chemotherapy for advanced pancreatic cancer. Main Outcomes and Measures: The main outcomes were the proportion of patients treated with each chemotherapy regimen over time and overall survival for each regimen. Cox proportional hazards regression models were used to compare overall survival between treatment regimens after adjustment for confounding variables, inverse probability of treatment weighting, and matching. Results: From 2008 to 2018, 5465 patients with advanced pancreatic cancer were treated with first-line chemotherapy in Ontario, Canada. The median (range) age of patients was 66.9 (27.8-93.4) years; 2447 (45%) were female; 878 (16%) had prior pancreatic resection, and 328 (6%) had prior adjuvant gemcitabine. During the time period when only gemcitabine and FOLFIRINOX were funded (2011-2015), 49% (929 of 1887) received FOLFIRINOX. When GEMNAB was subsequently funded (2015-2018), 9% (206 of 2347) received gemcitabine, 44% (1034 of 2347) received FOLFIRINOX, and 47% (1107 of 2347) received GEMNAB. The median overall survival increased from 5.6 months (95% CI, 5.1-6.0 months) in 2008 to 2011 to 6.9 months (95% CI, 6.5-7.4 months) in 2011 to 2015 to 7.6 months (95% CI, 7.1-8.0 months) in 2015 to 2018. Patients receiving FOLFIRINOX were younger and healthier than patients receiving GEMNAB. After adjustment and weighting, FOLFIRINOX was associated with better overall survival than GEMNAB (hazard ratio [HR], 0.75 [95% CI, 0.69-0.81]). In analyses comparing patients treated with GEMNAB and gemcitabine, GEMNAB was associated with better overall survival (HR, 0.86 [95% CI, 0.78-0.94]). Conclusions and Relevance: This cohort study of patients with advanced pancreatic cancer receiving first-line palliative chemotherapy within a universal health care system found that drug funding decisions were associated with increased uptake of new treatment options over time and improved survival. Both FOLFIRINOX and GEMNAB were associated with survival benefits in distinct patient populations.

Cost-effectiveness analysis of first-line treatment with crizotinib in ROS1-rearranged advanced non-small cell lung cancer (NSCLC) in Canada.

INTRODUCTION: While no direct comparative data exist for crizotinib in ROS1+ non-small cell lung cancer (NSCLC), studies have suggested clinical benefit with this targeted agent. The objective of this study was to assess the cost-effectiveness of crizotinib compared to standard platinum-doublet chemotherapy for first-line treatment of ROS1+ advanced NSCLC. METHODS: A Markov model was developed with a 10-year time horizon from the perspective of the Canadian publicly-funded health care system. Health states included progression-free survival (PFS), up to two further lines of therapy post-progression, palliation and death. Given a lack of comparative data and small study samples, crizotinib or chemotherapy studies with advanced ROS1+ NSCLC patients were identified and time-to-event data from digitized Kaplan-Meier curves were collected to pool PFS data. Costs of drugs, treatment administration, monitoring, adverse events and palliative care were included in 2018 Canadian dollars, with 1.5% discounting. An incremental cost-effectiveness ratio (ICER) was estimated probabilistically using 5000 simulations. RESULTS: In the base-case probabilistic analysis, crizotinib produced additional 0.885 life-years and 0.772 quality-adjusted life-years (QALYs) at an incremental cost of $238,077, producing an ICER of $273,286/QALY gained. No simulations were found to be cost-effective at a willingness-to-pay threshold of $100,000/QALY gained. A scenario analysis assuming efficacy equivalent to the ALK+ NSCLC population showed a slightly more favorable cost-effectiveness profile for crizotinib. CONCLUSIONS: Available data appear to support superior activity of crizotinib compared to chemotherapy in ROS1+ advanced NSCLC. At the list price, crizotinib was not cost-effective at commonly accepted willingness-to-pay thresholds across a wide range of sensitivity analyses.

Considerations for Developing a Reassessment Process: Report from the Canadian Real-World Evidence for Value of Cancer Drugs (CanREValue) Collaboration's Reassessment and Uptake Working Group.

The Canadian Real-world Evidence for Value in Cancer Drugs (CanREValue) Collaboration was established to develop a framework for generating and using real-world evidence (RWE) to inform the reassessment of cancer drugs following initial health technology assessment (HTA). The Reassessment and Uptake Working Group (RWG) is one of the five established CanREValue Working Groups. The RWG aims to develop considerations for incorporating RWE for HTA reassessment and strategies for using RWE to reassess drug funding decisions. Between February 2018 and December 2019, the RWG attended four teleconferences (with follow-up surveys) and two in-person meetings to discuss recommendations for the development of a reassessment process and potential barriers and facilitators. Modified Delphi methods were used to gather input. A draft report of recommendations (to December 2018) was shared for public consultation (December 2019 to January 2020). Initial considerations for developing a reassessment process were proposed. Specifically, reassessment can be initiated by diverse stakeholders, including decision makers from public drug plans or industry stakeholders. The reassessment process should be modelled after existing deliberation and recommendation frameworks used by HTA agencies. Proposed reassessment outcome categories include maintaining status quo, revisiting funding criteria, renegotiating price, or disinvesting. Overall, these initial considerations will serve as the basis for future advancements by the Collaboration.

Real-World Cost-Effectiveness of Bevacizumab With First-Line Combination Chemotherapy in Patients With Metastatic Colorectal Cancer: Population-Based Retrospective Cohort Studies in Three Canadian Provinces.

Background. Real-world evidence can be a valuable tool when clinical trial data are incomplete or uncertain. Bevacizumab was adopted as first-line therapy for metastatic colorectal cancer (mCRC) based on significant survival improvements in initial clinical trials; however, survival benefit diminished in subsequent analyses. Consequently, there is uncertainty surrounding the cost-effectiveness of bevacizumab therapy achieved in practice. Objective. To assess real-world cost-effectiveness of first-line bevacizumab with irinotecan-based chemotherapy versus irinotecan-based chemotherapy alone for mCRC in British Columbia (BC), Saskatchewan, and Ontario, Canada. Methods. Using provincial cancer registries and linked administrative databases, we identified mCRC patients who initiated publicly funded irinotecan-based chemotherapy, with or without bevacizumab, in 2000 to 2015. We compared bevacizumab-treated patients to historical controls (treated before bevacizumab funding) and contemporaneous controls (receiving chemotherapy without bevacizumab), using inverse-probability-of-treatment weighting with propensity scores to balance baseline covariates. We calculated incremental cost-effectiveness ratios (ICER) using 5-year cost and survival adjusted for censoring, with bootstrapping to characterize uncertainty. We also conducted one-way sensitivity analysis for key drivers of cost-effectiveness. Results. The cohorts included 12,112 (Ontario), 1,161 (Saskatchewan), and 2,977 (BC) patients. Bevacizumab significantly increased treatment costs, with mean ICERs between $78,000 and $84,000/LYG (life-year gained) in the contemporaneous comparisons and $75,000 and $101,000/LYG in the historical comparisons. Reducing the cost of bevacizumab by 50% brought ICERs in all comparisons below $61,000/LYG. Limitations. Residual confounding in observational data may bias results, while the use of original list prices overestimates current bevacizumab cost. Conclusion. The addition of bevacizumab to irinotecan-based chemotherapy extended survival for mCRC patients but at significant cost. At original list prices bevacizumab can only be considered cost-effective with certainty at a willingness-to-pay threshold over $100,000/LYG, but price reductions or discounts have a significant impact on cost-effectiveness.

Significantly Minimizing Drug Wastage and the Cost of Cabazitaxel Used to Treat Metastatic Castration-Resistant Prostate Cancer.

Cabazitaxel is used to treat patients with metastatic castration-resistant prostate cancer progressing after docetaxel. It is prepackaged in 60 mg single-dose vials, a quantity much higher than the average prescribed dose, which leads to, substantial drug wastage (DW) and associated costs. To minimize DW we implemented a cost-saving, cohorting strategy where multiple patients scheduled to receive cabazitaxel (at a dose of 20mg/m2 every 3 wks) were cohorted and treated on a single weekday whenever possible. Excess drug from each vial was then saved and used for subsequent patients treated on the same day. The drug cost with cohorting was calculated from the actual number of vials used, and the drug cost without cohorting was estimated by assumingthat one vial was used per treatment. The cost of DW was determined based on the amount of drug that was discarded. All cost calculations also accounted for the discount incentives offered by Sanofi-Aventis. Over a 3-yr period, 74 patients received 402 treatments of cabazitaxel. Multiple patients were treated on 67.4% of the treatment days, and grouping of three patients on one day saved one vial. The estimated total drug cost saved was $394 536 CAD (21.1%). Pending further studies on safety and efficacy, this strategy could potentially be adopted to mitigate DW for cabazitaxel and similarly for other oncology drugs. This would significantly decrease the overall financial burden on patients, institutions, and stakeholders. PATIENT SUMMARY: Cabazitaxel chemotherapy is associated with substantial drug wastage and associated costs. By cohorting patients scheduled to receive cabazitaxel on a single weekday, the total drug cost was decreased by $394 536 CAD (21.1%) over a 3-yr period. Similar strategies could be considered to overcome the prohibitory costs associated with drug wastage for cabazitaxel and other cancer drugs.

Real-world, population-based cohort study of toxicity and resource utilization of second-line ipilimumab for metastatic melanoma in Ontario, Canada.

Second-line ipilimumab has been publicly funded in Ontario for metastatic melanoma (MM) since September 2012. We examined real-world toxicity of second-line ipilimumab compared to standard second-line treatments prior to funding. MM patients who received systemic treatment from April 2005 to March 2015 were included. Patients receiving second-line ipilimumab after September 2012 were considered as cases, and those who received second-line treatment prior to the funding date were included as historical controls. Outcomes assessed include treatment-related mortality, any-cause hospital visits, ipilimumab-related hospital visits and specialist visits (eg, endocrinologists, ophthalmologists, gastroenterologists, rheumatologists and respirologists), which were captured from up to 30 and/or 90 days after end of second-line treatment. Inverse probability of treatment weighting was used to adjust for baseline differences between groups. Odds ratios (ORs) from logistic regressions and rate ratios (RRs) from rate regressions were used to assess differences between groups. We identified 329 MM patients who received second-line treatments (ipilimumab: 189; controls: 140). Ipilimumab was associated greater any-cause (60.1% vs 45.7%; OR = 1.81; P value = .019) and ipilimumab-related (47.2% vs 31.9%; OR = 1.91; P value = .011) hospital visits. Adjusting for different follow-up days, ipilimumab was associated with higher rates of all-cause (RR = 1.56 [95%CI: 1.12-2.16]), and ipilimumab-related (RR = 2.18 [95% CI: 1.45-3.27]) hospital visits. Patients receiving ipilimumab were more likely to visit specialist involved in immunotherapy toxicity management (23.5% vs 13.7%; P value = .04). Compared to historical second-line treatments, second-line ipilimumab was associated with more health service utilization (specifically hospital visits and specialist visits), suggestive of potentially increased toxicity in the real world.

Use of real-world evidence in cancer drug funding decisions in Canada: a qualitative study of stakeholders' perspectives.

BACKGROUND: Real-world evidence (RWE) can provide postmarket data to inform whether funded cancer drugs yield expected outcomes and value for money, but it is unclear how to incorporate RWE into Canadian cancer drug funding decisions. As part of the Canadian Real-World Evidence Value for Cancer Drugs (CanREValue) Collaboration, this study aimed to explore stakeholder perspectives on the current state of RWE in Canada to inform a Canadian framework for use of RWE in cancer drug funding decisions. METHODS: This was a qualitative descriptive study. Qualitative semistructured interviews were conducted from April to July 2018. Participants were Canadian and international stakeholders who had experience with RWE and drug funding decision-making. Thematic analysis was used to analyze data. RESULTS: Thirty stakeholders participated in the study. Five themes were identified. Stakeholders indicated that RWE had value in cancer drug funding decisions. However, a cultural shift is needed to adopt RWE in decision-making. Further, the Canadian infrastructure for real-world data is currently inadequate for decision-making, and there is a need for committed investment in building capacity to collect and analyze RWE. Finally, there is a need for increased collaboration among key stakeholders. INTERPRETATION: The findings of this study suggest that if RWE is to be used in drug funding decisions, there is a need for a cultural shift, improved data infrastructure, committed investment in capacity building and increased stakeholder collaboration. Together with local stakeholder engagement, application of these findings may contribute to optimizing implementation of RWE.

The effectiveness of rituximab and HIV on the survival of Ontario patients with diffuse large B-cell lymphoma.

INTRODUCTION: For patients with diffuse large B-cell lymphoma (DLBCL), standard-care is rituximab administered with CHOP or CHOP-like chemotherapy (R-CHOP). However, the effectiveness and safety of R-CHOP among DLBCL patients with human immunodeficiency virus (HIV) infection is less clear, as HIV+ patients were omitted from most clinical trials and population-level data from unselected patients are limited. R-CHOP was funded for HIV-associated DLBCL patients with CD4 >50/mm3 in Ontario in February 2015. METHODS: Patients with a new diagnosis of DLBCL were identified from the Ontario Cancer Registry between April 2010 and March 2018. HIV diagnosis and chemotherapy regimen were ascertained using administrative databases at Ontario Health. The effect of rituximab and HIV on overall survival was assessed in the HIV+ subgroup (R-CHOP vs CHOP) and in the R-CHOP subgroup (HIV+ vs HIV-). RESULTS: Among HIV+ patients, receipt of R-CHOP was associated with a fivefold improvement in overall survival (hazard ratio [HR] 0.29 (0.13-0.66) compared with CHOP), after adjustment. Among patients who received R-CHOP (n = 6106), older age, male sex, lower neighborhood income, and higher comorbidity were associated with worse overall survival, after adjustment (P < .001 for all), but HIV positivity was not prognostic (HR 1.12 (0.60-2.10)). Within 1-year after diagnosis, HIV+ patients receiving R-CHOP had a similar proportion of patients who visited the emergency department (67% vs 66% P = .43) or admitted to hospital (58% vs 52%, P = .43) and as HIV- patients receiving R-CHOP. CONCLUSION: HIV status did not affect prognosis for patients with DLBCL receiving R-CHOP in an unselected general population when rituximab was used according to funding criteria. R-CHOP was safe and effective for DLBCL treatment, regardless of HIV status.

Impact of rarity on Canadian oncology health technology assessment and funding.

OBJECTIVES: The pan-Canadian Oncology Drug Review (pCODR) evaluates new cancer drugs for public funding recommendations. While pCODR's deliberative framework evaluates overall clinical benefit and includes considerations for exceptional circumstances, rarity of indication is not explicitly addressed. Given the high unmet need that typically accompanies these indications, we explored the impact of rarity on oncology HTA recommendations and funding decisions. METHODS: We examined pCODR submissions with final recommendations from 2012 to 2017. Incidence rates were calculated using pCODR recommendation reports and statistics from the Canadian Cancer Society. Indications were classified as rare if the incidence rate was lower than 1/100,000 diagnoses, a definition referenced by the Canadian Agency for Drugs and Technologies in Health. Each pCODR final report was examined for the funding recommendation/justification, level of supporting evidence (presence of a randomized control trial [RCT]), and time to funding (if applicable). RESULTS: Of the ninety-six pCODR reviews examined, 16.6 percent were classified as rare indications per above criteria. While the frequency of positive funding recommendations were similar between rare and nonrare indication (78.6 vs. 75 percent), rare indications were less likely to be presented with evidence from RCT (50 vs. 90 percent). The average time to funding did not differ significantly across provinces. CONCLUSION: Rare indications appear to be associated with weaker clinical evidence. There appears to be no association between rarity, positive funding recommendations, and time to funding. Further work will evaluate factors associated with positive recommendations and the real-world utilization of funded treatments for rare indications.

Real-world comparative effectiveness of second-line ipilimumab for metastatic melanoma: a population-based cohort study in Ontario, Canada.

BACKGROUND: For novel cancer treatments, effectiveness in clinical practice is not always aligned with clinical efficacy results. As such it is important to understand a treatment's real-world effectiveness. We examined real-world population-based comparative effectiveness of second-line ipilimumab versus non-ipilimumab treatments (chemotherapy or targeted treatments). METHODS: We used a cohort of melanoma patients receiving systemic treatment for advanced disease since April 2005 from Ontario, Canada. Patients were identified from provincial drug databases and the Ontario Cancer Registry who received second-line ipilimumab from 2012 to 2015 (treated) or second-line non-ipilimumab treatment prior to 2012 (historical controls). Historical controls were chosen, to permit the most direct comparison to pivotal trial findings. The cohort was linked to administrative databases to identify baseline characteristics and outcomes. Kaplan-Meier curves and multivariable Cox regression models were used to assess overall survival (OS). Observed potential confounders were adjusted for using inverse probability of treatment weighting (IPTW). RESULTS: We identified 329 patients with metastatic melanoma (MM) who had received second-line treatments (189 treated; 140 controls). Patients receiving second-line ipilimumab were older (61.7 years vs 55.2 years) compared to historical controls. Median OS were 6.9 (95% CI: 5.4-8.3) and 4.95 (4.3-6.0) months for ipilimumab and controls, respectively. The crude 1-year, 2-year, and 3-year OS probabilities were 34.3% (27-41%), 20.6% (15-27%), and 15.2% (9.6-21%) for ipilimumab and 17.1% (11-23%), 7.1% (2.9-11%), and 4.7% (1.2-8.2%) for controls. Ipilimumab was associated with improved OS (IPTW HR = 0.62; 95% CI: 0.49-0.78; p < 0.0001). CONCLUSIONS: This real-world analysis suggests second-line ipilimumab is associated with an improvement in OS for MM patients in routine practice.